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Basic Assumption 


€ We will focus on TCP/IP and Application layer 
protocols 

€ We will focus on IPv4 

€ You are comfortable with TCP/IP host configuration 
& hardware 

9 Only dealing with software based network sniffers 

€ Will be covering debugging techniques based on real 
examples 
a will not cover every possible network error, bug, or glitch 
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Justification 


€ Software debugging techniques 

4 printf() 

4 symbolic debuggers (e.g. gdb) 

€ Network application debugging techniques 

4 log files 

3 raw packets & network protocol analyzers 

€ Sometimes you don't have access to the application 
logs or they are just wrong 

9 Don't overlook bad hardware, device drivers, or 
application bugs 

€ No magic bullet, just another tool in the utility belt 
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Outline 


O Ethereal 
4 Overview and Installation 
4 Captures & filters 


€ TCP/IP Networking 
€ Application Protocols 
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Packet Sniffers 


€ Tools which enable record, monitor, or analyze 
network traffic 

€ Network Monitoring Tools 

3 http://www.slac.stanford.edu/xorg/nmtf/nmtf-tools.html 

€ Software based 

3 tcpdump, ethereal, snoop, etc... 

€ Hardware based 
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Ethereal 


9 http://www.ethereal.com/ 

9 Available for Unix, Windows, and OS X 

€ Components 

3 ethereal — gtk+ based GUI 

4 tethereal — command line version of GUI 

4 editcap, mergecap, text2pcap, capinfos — command line 
tools for manipulating capture files 

€ Many supported file formats including pcap, MS' 
netmon, & Solaris' snoop 
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Dependencies 
€ Required 
4 glib 2.0, gtk+ 2.0 
€ Optional 


4 GNU ADNS library 

Perl Compatible Regular Expression library 
Zlib 

Net-SNMP libs 

Kerberos and OpenSSL 


OCCO O 
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Building.... 


$ ./configure --perfix=/opt/ethereal --with-krb5=/usr -with-ssl 
The Ethereal package has been configured with the following 


options. 


Build ethereal : yes 
Build tethereal : yes 
Install setuid : no 

Use plugins : yes 


Use GTK+ v2 library : yes 


Use threads : no 


Build profile binaries : no 


Use pcap library : yes 
Use zlib library : yes 
Use pcre library : yes 


Use kerberos library : yes (MIT) 
Use GNU ADNS library : no 


Use SSL crypto library : yes 
Use IPv6 name resolution : yes 
Use UCD SNMP/Net-SNMP library : no 
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Screenshot 


$ ethereal 


capture 


decoded 
packet 


raw data 


@ centeris 


* (Untitled) - Ethereal ry 
Edit View Go Capture Analyze Statistics Help 


SB*@8 Nar DTZ aaa emn 


ax © 


DE >| + Expression... |% clear| v Appiy| 
No. - [Time Source Destination ?rotocol] Info 
57.170828 Vmware 8b:bf:ef Broadcast ARP Who has 192.168.1.144? Tell 192.168.1.101 
€ 8.431791 —192.168.1.178 130.239.18.172 IRC Request 
78.622197 130.239.18.172 192.168.1.178 TCP 6667 > 33147 [ACK] Seq=0 Ack=20 Win=1448 Len=0 TSV=614344665 TSER=1 
8 8.657783 — 130.239.18.172  192.168.1.178 IRC Response 
9 8.657871 192.168.1.178 130.239.18.172 TCP 33147 > 6667 [ACK] Seq=20 Ack=62 Win=16022 Len=0 TSV=19090432 TSER= 


38002 > http Seq=0 Ack=0 Win=5840 Len=0 MSS=1460 TSV=1909145: 
11 9.724711  66.70.73.150 192.168.1.178 TCP http > 38002 [SYN, ACK] Seq-0 Ack=1 Win=5792 Len=0 MSS=1460 Ti 
12 9.724845 192.168.1.178 66.70.73.150 TCP 38002 > http [ACK] Seq=1 Ack=1 Win=5840 Len=0 TSV=19091499 TSER=32C || 


b Frame 10 (74 bytes on wire, 74 bytes captured) 
b Ethernet II, Src: 00:30:ab:05:£7:61, Dst: 00:00:f4:d8:37:d0 
upam A E 1178) E EE 
> Transmission Control Protocol, Sre Port: 38002 (38002), Dst Port: http (80), Seq: 0, Ack: 0, Len: 0 
Source port: 38002 (38002) 
et en 
Sequence number: 0 (relative sequence number) 
Header length: 40 bytes 
| elas: 050002 (5 
Window size: 5840 
Checksum: Ox5f0b (correct) 
v Options: (20 bytes) 
Maximum segment size: 1460 bytes 
SACK permitted 
star Peat LA QU a PING eq 
Be 
Window scale: 2 (multiply by 4) 


[0000 00 00 fi d8 37 dO 00 30 ab 05 £7 61 08 00 45 00 
010 00 3c 24 62 40 00 40 06 c8 23 c0 a8 01 b2 42 46 
0020 49 96 94 72 00 50 32 71 6b 9f 00 00 00 00 a0 
0030 16 do 5f 0b 00 00 02 04 05 b4 04 02 08 Oa 01 23 
0040 4f fd 00 00 00 00 01 03 03 02 


Flags (tcp.flags)... JP: 192 D: 192 M: 0 
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Capture Dialog 


CTRL-K * 


Capture 


Ethereal: Capture Options "x 


Interface: [wiano 


IP address: 192.168.1.178, fe80::230:abff:fe05:f761 
Link-layer header type: Ethernet) > 


F Capture packets in promiscuous mode 
T Limit each LE to|68 -| bytes 


Capture File(s) 


File: Browse... 


F Use multiple files 


ring 


F Next file every 


buffer 


=| minute(s) 


E: 


=| megabyte(s)| = 


1 

F Next file every 1 
F Ring buffer with |2 
1 


| files 


F Stop capture after 


Stop Capture ... 


F... after 1 


F... after 1 


V... after 1 
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=| file(s) 


-| packet(s) 
=| megabyte(s) t 


|- minute(s) 


Name Resolution 


Display Options 
F Update list of packets in real time 
F Automatic scrolling in live capture 


F Hide capture info dialog 


F Enable MAC name resolution 


T Enable network name resolution 


F Enable transport name resolution 


interface, 
capture filter, 
snap length, 
etc... 


DNS lookups, 
automatic 


. display, etc... 


X Cancel| OK | 
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Interfaces 
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€ Any configured network interface is available 


4 ifconfig -a 
4 tethereal -D 
4 netstat -n -i 


9 any 


4 view packets on all interfaces 


e eid length (a.k.a. snap length) 


link layer 


header 
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Copyright Gerald 


number of bytes of the packet that should be stored 
4 generally limit this based on the max transmit unit of the 


4 ethernet MTU is 1500 bytes of data + 14 bytes of frame 


Carter, 2005-2006. All rights reserved 
jerry@samba.org, Slide 12 


Ring Buffers 


O A ring buffer is a set of N files of max size Z 

4 Ethereal will write to the first file until a defined 
condition is met and then move onto the next 

4 When the last file is full, ethereal rolls over to the first 
file 

9 End of file conditions 

4 size 

a time 

9 Multiple files can be merged (mergecap) and then 
filtered from a command line using tethereal 
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Ethereal & Filters 


€ Two types of filters 

4 capture filters 

4 display filters 

€ Capture filters use the libpcap filter facility 

3 man tcpdump(1) 

4 e.g. “port 80 and host 192.168.1.100” 

9 Display filters are built upon the ethereal protocol 
dissectors 


3 supports referencing protocol components by name 
3 e.g. “http && ip.addr == 192.168.1.100” 
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Capture Filters 


O expression 

a type (host, net, port) 

3 direction (src, dst) 

3 protocol (ether, arp, tcp, ... ) 

9 logical operators 

2 not(!) 

3 and(&&) 

a or (||) 

€ Name resolution supported by DNS, /etc/services, 
/etc/networks, etc... 

9€ Enclose capture expression in quotes if using any 
special shell characters (e.g. > or !) 
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Capture Filter Examples 


€ Capture all packets from the machine at 192.168.1.1 

4 “sre host 192.168.1.1" 

€ Capture all Address Resolution Packets 

3 “arp” 

€ Capture all packets to and from the host 
worm.plainjoe.org where the source or destination 
port is 80 


4 “tcp port 80 and host worm.plainjoe.org” 
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Extracting Bytes 


€ Certain protocols can be referenced as arrays 

3 value returned in network byte order (big endian) 

2 tcp[0] means the first byte in the tcp packet 

4 tcp[0:2] returns a 2 byte value 

4 tcp[0:4] returns a 4 byte value 

O Bitwise operators & and | are supported 

O Basic set of C relational operators 

Lp < < > >= edu 

€ Example: capture ping packets 

4 icmp[0] == 0x8 or icmp[0] == 0x0 

4 icmp[icmptype] == icmp-echo or 
icmp[icmptype] == icmp-echoreply 
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Display Filters 


9 Display filters allow comparisons based on most 
fields on the protocol dissector 
a Help -> Supported Protocols 

€ Used to filter existe ey denne color rules, or 


filter statistics ETT 
Field name Relation Value (Boolean) 
E Can be entered tcp.flags.cwr - Congestion A fis present 1 
2 7 tep.flags.ecn - ECN-Echo == | Predefined values: 
directly or built tepagsurg-urgert I= 
tcp.flags.ack - Acknowledg Not set 
: d : l tcp.flags. push - Push 
via a dialog Bog. 
tcp.flags.syn - Syn 
tcp.flags.fin - Fin T 
tcp.window_size - Window Range (offset:length) 
4 — == 


X Cancel 


show all tcp reset packets 
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Display Filter Operators 


Operator Description 


> or gt greater than 
> or ge than or equal to 
« or It less than 
<=or le less than or equal to 
== or eq equal 
!= or ne not equal 
contains string (or byte) search 
matches regular expression match (pcre) 


Display Filter Data Types 


Numerical 

3 int, float, .... 

Strings 

4 characters, bytes, .... 
Addresses 

4 network, hardware, .... 
Time 


4 absolute time and relative time between packets 
Protocol keywords 
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Examples 


€ spoolss && dcerpc.opnum == 0x45 

3 Win32 OpenPrinterEx() 

9 ip.addr == 192.168.1.1 

4 ip.src == 192.168.1.1 || ip.dst == 192.168.1.1 
9 ip.flags.df 

3 IP packets with the “Don't Fragment" bit set 

O http contains “POST” 

4 search for the string “POST” in all http packets 
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Display Filter Ranges 


€ Ranges allow for using slices of byte sequences in 
filter expressions 
Range Description 
[offset] single byte at offset 
[offset:N] N bytes starting at optional offset 
[offset - offset] bytes from offset, to offset, inclusive 
[offset:] bytes in field starting at offset 
[range,range | combine multiple ranges 
9€ Find all HTTP packets on 192.168.1.0/24 
a http && (ip[12:3] == c0:a8:01 && ip[16:3] == c0:a8:01) 
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Saving Filters 


€ Both capture and display filters [em me 7 7s 
can be saved in the user ,,/ — 
preferences 

€ Stored in ~/.ethereal/ ces 
ud cfilters Dant local http traffic 
3 dfilters Filter string: [http && (ip[12:3] = + Expression... | 
mu colorfilters (Help | save | ¥ Apply | X Close | ok 
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Command line captures 


9 tethereal 
-h (help text) 
-w outfile (write to file) 
-s snaplen (capture snaplen bytes of packet) 
-i interface (name of NIC to watch) 
-n (disable name resolution) 
-b (multiple file mode) 
v files:numfiles (number of files in ring buffer) 
v filesize:Kb (swap to next file after N Kb) 
v duration:seconds (swap to next file after M seconds) 
-à (criteron to move to next file; same options as -b) 
3 -f*capture filter” 
v can also pass mult-expression filters as last arguments 
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L 


COCO Ù 


Multiple Capture Files 


€ Multiple files (until disk is full) 

3 tethereal -b -a {duration, filesize, files) :value -w x.pcap 
@ Ring buffer 

4 tethereal -b files:num -a test:value -w x.pcap 

€ Merging and filtering files 

3 mergecap -w newfile.pcap x*.pcap 

4 tethereal -r newfile.pcap -R dilter -w filtered.pcap 
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Popular CLI Sniffers 


€ Operating systems often come with a command line 
packet capture tool installed 

4 frequently easier to use an installed tool than deploying a 

new package 

4 noring buffer support in general 

€ Solaris' snoop 

3 snoop -r -o /tmp/dump.snoop -d hme cfilter 

9 tcpdump 

3 similar arguments as tethereal 

3 tcpdump -w /tmp/dump.pcap -s 0 -i ethO cfilter 
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This slide intentionally left blank. 
--anonymous 
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TCP/IP Protocol 


@ centeris 
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OSI vs. TCP/IP 


OSI 


Application 


TCP/IP 


Presentation 


Session 


Transport 


Application 


Network 


Transport 


Data Link 


Network 


Physical 
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Link 
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TCP/IP Protocol Suite 


Applications (Samba, nfsd, Apache, Postfix, BGP ,...) 


TCP UDP 
ICMP |. — IP 
| 
p Etherenet, Token 
kg Leder Ring, etc.... 
| 
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Address Resolution Protocol 


€ Arp (“What's the ethernet address for aa.bb.cc.dd?”) 
2 Matches network addresses to link layer addresses using 
the link broadcast address 


€ Reverse Arp (“What's the IP address for this ethernet 
address?") 
a commonly used for diskless booting 

€ Gratuitous Arp (“Who has aa.bb.cc.dd? Tell 
aa.bb.cc.dd.”) 
4 Frequently used by hosts when booting to detect a 

duplicate IP address on the network 
€ Proxy Arp 


Q Used by routers to answer Arp requests for remote hosts 
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Failed Ping 


rain$ ping 192.168.1.1 2>&1 > /tmp/ping.log & 


rain$ tethereal -i wlanO \ 

ether host 00:30:AB:05:F7:61 and arp 

Capturing on wlan0 

0.000000 00:30:ab:05:£7:61 -> £f:ff:ff:ff:ff:ff 
ARP Who has 192.168.1.1? Tell 192.168.1.178 

0.999845 00:30:ab:05:£7:61 -> £ff:ff:ff:ff:ff:ff 
ARP Who has 192.168.1.1? Tell 192.168.1.178 


snow$ tethereal -n -i eth0 \ 

ether host 00:30:AB:05:F7:61 and arp 

Capturing on eth0 

51.655889 00:30:ab:05:£7:61 -> ff:ff:ff:ff:ff:ff 
ARP Who has 192.168.1.1? Tell 192.168.1.178 

51.655958 00:00:£4:d8:37:d0 -> 00:30:ab:05:£7:61 
ARP 192.168.1.1 is at 00:00:£4:d8:37:d0 


nr 
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Duplicate IP 


rain$ Å tethereal -i wlanO \ 
ether host 00:30:AB:05:F7:61 and arp 
Capturing on wlan0 
1.514118 00:30:ab:05:£7:61 -> ff:ff:ff:ff:ff:ff 

ARP Who has 192.168.1.71? Tell 192.168.1.178 
1.516633 00:0c:29:a9:b5:2f -> 00:30:ab:05:£7:61 

ARP 192.168.1.71 is at 00:0c:29:a9:b5:2£ 
1.517694 00:0c:29:63:18:bd -> 00:30:ab:05:£7:61 

ARP 192.168.1.71 is at 00:0c:29:63:18:bd 


DP 
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IP Datagrams 


€ Delivered host to host 

€ Destination host kernel demultiplexes to TCP/IP 
applications based on port numbers specified in the 
transport layer 


0 ~« » 15 16 a »31 
4-bit 4-bit 
version | hdr len TOS total length 
3-bit 
identification flags fragment offset 
TTL protocol checksum 


source IP address 


destination IP address 


IPv4 Header 
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UDP 


€ Connectionless, no ACK, retransmits up to 
application 

@ Single packet payload 

4 theoretically limited by 16-bit length 

4 generally <= 8200 bytes (8192 + 8 byte header) 


0 a p» 15 16 a »-31 


source port destination port 


length checksum 


UDP Header 
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Unserviced UDP Ports 


$ host foo 192.168.1.1 
;; connection timed out; no servers could be reached 


Capturing on wlan0 
0.000000 192.168.1.178 -> 192.168.1.1 

DNS Standard query A foo.plainjoe.org 
0.002487 192.168.1.1 -> 192.168.1.178 


5.001193 192.168.1.178 -> 192.168.1.1 
DNS Standard query A foo.plainjoe.org 
5.003729 192.168.1.1 -> 192.168.1.178 


$ tethereal -p -i wlan0 arp or icmp or port 53 


ICMP Destination unreachable (Port unreachable) 


ICMP Destination unreachable (Port unreachable) 
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Internet Control Message Protocol 


€ ICMP is used to communicate 
4 error message (e.g. host unreachable) 
4 status information (e.g. ping) 


9€ Carried in IP packets 


3 network information (e.g. redirect for network) 
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ICMP: Port Unreachable 


€ Returned when a UDP packet is sent to a port which 
has no listening process 


0 ~« > 15 16 a »31 


type code checksum 


20 byte IP header + at least 8 bytes of UDP packet 


ICMP: Port Unreachable 
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Traceroute 


€ Use TTL-exceeded ICMP error to map path from 
source to destination 

O Starts with a UDP packet using a TTL of 1 and 
increments monotonically 

€ Router returns time-to-live exceeded ICMP message 
when the TTL (after decrementing) reaches 0 
3 TTL of 1 should map 1* hop, TTL of 2 should map 2" 

hop, etc... 

O Linux traceroute sends out 3 UDP probes per hop to 
log round trip time (RTT) 

9 Beware of firewalls 
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Traceroute Example 


$ traceroute -n -N 1 plainjoe.org 

traceroute to plainjoe.org (69.30.213.223), 30 hops max, 40 byte packets 
1 192.168.1.1 3.914 ms 3.074 ms 3.133 ms 
2 10.108.32.1 9.218 ms 11.440 ms 14.474 ms 


$ tethereal -i wlan0 udp or icmp 
Capturing on wlan0 
0.000000 192.168.1.178 -> 69.30.213.223 

UDP Source port: 64000 Destination port: traceroute 
0.002540 192.168.1.1 -> 192.168.1.178 

ICMP Time-to-live exceeded (Time to live exceeded in transit) 
0.034940 192.168.1.178 -> 69.30.213.223 

UDP Source port: 64001 Destination port: 33435 
0.037540 10.108.32.1 -> 192.168.1.178 

ICMP Time-to-live exceeded (Time to live exceeded in transit) 
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TCP 


€ Connection oriented, acknowledgments, 
retransmission, byte stream, etc... 


€ Fragments byte stream into TCP segments 


0 a > 15 16 < »31 


source port destination port 


Sequence number 


acknowledgement number 


4-bit 6-bit “eae i : 
hdrlen| reserved | , flags window size 
checksum urgent pointer 
TCP Header 
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TCP 3-way Handshake 


€ Client sends SYN packet with initial sequence 
number 

@ Server responds with a SYN/ACK 
4 ACK number is ISN+1 

9€ Client responds with ACK 


Time 0:00 
Client | —- Server 


sm 


de 


AC 


ESTABLISHED 
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4 Way Close 


€ FIN bit is for normal TCP session termination 
€ TIME WAIT lasts for 2 x MSL 


I max segment Client Time 0:00 
lifetime FIN WAITI Ry Server 
| ^» CLOSE WAIT 
NE [LAST ACK 
FIN WAIT2 a pe 
= $m 

TIME WAIT | — 
| Ck 
| LN 

2MSL , CLOSED 
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netstat 


$ # netstat -n 
Active Internet connections (w/o servers) 


Proto Recv-Q Send-Q Local Address Foreign Address State 

tcp 0 0 127.0.0.1:35400 127.0.0.1:4993 CLOSE WAIT 
tcp 0 0 192.168.1.178:35221 66.70.73.150:22 ESTABLISHED 
tcp 0 0 127.0.0.1:4993 127.0.0.1:35399 FIN WAIT2 
tcp 0 0 127.0.0.1:4993 127.0.0.1:35401 FIN WAIT2 
tcp 0 0 127.0.0.1:4993 127.0.0.1:35400 FIN WAIT2 
tcp 0 0 192.168.1.178:39701 192.168.1.56:143 ESTABLISHED 
tcp 0 0 192.168.1.178:34259 192.168.1.56:143 CLOSE WAIT 
tcp 0 0 192.168.1.178:34260 192.168.1.56:143 CLOSE_WAIT 
tcp 0 0 192.168.1.178:36959 192.168.1.84:22 TIME WAIT 
tcp 0 0 192.168.1.178:38788 192.168.1.84:22 ESTABLISHED 
tcp 0 0 127.0.0.1:6010 127.0.0.1:35979 ESTABLISHED 
tcp 0 0 127.0.0.1:49677 127.0.0.1:631 CLOSE WAIT 
tcp 0 0 192.168.1.178:46140 192.168.1.84:22 ESTABLISHED 
tcp 0 0 192.168.1.178:51979 192.168.1.44:22 ESTABLISHED 
tcp 896 0 127.0.0.1:59245 127.0.0.1:22 ESTABLISHED 
tcp 0 0 127.0.0.1:35979 127.0.0.1:6010 ESTABLISHED 
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Unserviced TCP Ports 
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telnet: 


$ telnet 192.168.1.1 53 
Trying 192.168.1.1... 
connect to address 192.168.1.1: 


Connection refused 


$ tethereal -p -n -i wlan0 tcp and port 53 


Capturing on wlan0 

7.787969 192.168.1.178 -> 192.168.1.1 TCP 53329 > 53 [SYN] 
Seq=0 Ack=0 Win=5840 Len=0 MSS=1460 TSV=31781951 
TSER=0 WS=2 

7.790345 192.168.1.1 -> 192.168.1.178 TCP 53 > 53329 [RST, ACK] 
Seq=0 Ack=0 Win=0 Len=0 
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TCP SYN Tricks 


€ SYN Scan 
4 send SYN packets looking for open ports 
d eg. 
v “closed” ports return [RST,ACK] 
v “open” ports return [SYN,ACK] 
v "filtered" ports are protected by a firewall 
€ SYN Attacks 
4 Client sends a SYN but does not send the final ACK 
leaving a half open connection 
3 consumes resources in the server's connection queue 
4 syn cookies use a special ISN to protect against 
v  http:;//cr.yp.to/syncookies.html 
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This slide intentionally left blank. 
--anonymous 
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Application Protocols 
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Domain Name Service 


€ Utilizes both UDP and TCP port 53 

€ Normal queries are via UDP 

€ Client should retry using TCP when the UDP reply 
contains the "truncated" bit 

€ Already seen on instance of DNS timeouts caused by 
misconfigured /etc/resolv.conf 
Q client resolver continued to send name query even when 


the UDP packet elicited a “port unreachable” ICMP 
message 
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Scenario 


$ grep software /etc/auto.misc 

software -rw,hard,intr worm: /export/u2/software 
$ cd /misc/software 

cd: /misc/software: No such file or directory 


Oct 29 14:39:16 rain automount[6315]: 
attempting to mount entry /misc/software 
Oct 29 14:39:16 rain automount[13018]: 
mount(nfs): entry software: host worm: lookup failure 


# tethereal -n -i wlan0 icmp or port 53 
Capturing on wlan0 
0.000000 192.168.1.178 -> 192.168.1.56 

DNS Standard query A worm.painjoe.org 
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Scenario 


$ ping sleet.everest.plainjoe.org 
ping: unknown host sleet.everest.plainjoe.org 


; zone file for everest.plainjoe.org 


; Addresses for local hosts 


localhost IN A 127.0.0.1 
abydos IN A 192.168.1.54 
sleet IN A 192.168.1.41 
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Trace 


# tethereal -n -i wlan0 icmp or port 53 
Capturing on wlanO 
0.000000 192.168.1.178 -> 192.168.1.56 
DNS Standard query A sleet.everest.plainjoe.org 
0.003927 192.168.1.56 -> 192.168.1.178 
DNS Standard query response A 192.168.1.41 
5.002181 192.168.1.178 -> 192.168.1.56 
DNS Standard query A sleet.everest.plainjoe.org 
5.006624 192.168.1.56 -> 192.168.1.178 
DNS Standard query response A 192.168.1.41 
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DNS Reply Packet 


Domain Name System (response) 
Transaction ID: Oxde76 
Flags: 0x8780 (Standard query response, No error) 
al. .... .... = Truncated: Message is truncated 
0000 = Reply code: No error (0) 


Questions: 1 
Answer RRs: 1 
Authority RRs: 21 
Additional RRs: 0 
Queries 
sleet.everest.plainjoe.org: type A, class IN 
Answers 
sleet.everest.plainjoe.org: 
type A, class IN, addr 192.168.1.41 
Name: sleet.everest.plainjoe.org 
Type: A (Host address) 
Class: IN (0x0001) 
Time to live: 1 day 
Data length: 4 
Addr: 192.168.1.41 


@ centeris ran 


DNS Delays? 


$ cat /etc/resolv.conf 
domain ad.plainjoe.org 
nameserver 192.168.1.101 


$ host worm.plainjoe.org 
connection timed out; no servers could be reached 


VAD 


$ tethereal -n -i wlan0 port 53 
Capturing on wlan0 
0.000000 192.168.1.178 -> 192.168.1.101 
DNS Standard query A worm.plainjoe.org 
5.001037 192.168.1.178 -» 192.168.1.101 
DNS Standard query A worm.plainjoe.org 
15.191964 192.168.1.101 -> 192.168.1.178 
DNS Standard query response, Server failure 
20.200041 192.168.1.101 -> 192.168.1.178 
DNS Standard query response, Server failure 
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Follow the Bread Crumbs 


$ host -r worm.plainjoe.org 192.168.1.101 
37 nothing 


$ host -r -t NS plainjoe.org 192.168.1.101 
plainjoe.org name server thorn.plainjoe.org. 


$ host -r thorn.plainjoe.org 192.168.1.101 
thorn.plainjoe.org has address 192.168.1.56 


$ host worm.plainjoe.org 192.168.1.56 
worm.plainjoe.org has address 192.168.1.79 
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Tracing Windows DNS 


c:\> tethereal -r dns.pcap -R “ip.addr == 192.168.1.56" 
4 0.011988 192.168.1.101 -> 192.168.1.56 
DNS Standard query A worm.plainjoe.org 
6 5.002069 192.168.1.101 -> 192.168.1.56 
DNS Standard query A worm.plainjoe.org 
7 5.424241 192.168.1.101 -> 192.168.1.56 
DNS Standard query A worm.plainjoe.org 
13 10.424791 192.168.1.101 -> 192.168.1.56 
DNS Standard query A worm.plainjoe.org 
15 10.979990 192.168.1.101 -> 192.168.1.56 
DNS Standard query AAAA phzzbt.plainjoe.org 
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FTP 


€ Separate streams for control and data 
9 Originally the server performed an active open back 
to the client 


a client sent PORT command to notify the server regarding 
which port to connect to 


€ Newer passive mode requires the client to issue the 


active open 
a server sends the port number in the PASV reply 
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FTP PORT Command 


$ tethereal -n -i wlan0 host 192.168.1.56 

0.003808 192.168.1.178 -> 192.168.1.56 TCP 59398 > 21 [SYN] 
0.006091 192.168.1.56 -> 192.168.1.178 TCP 21 > 59398 [SYN, ACK] 
0.006128 192.168.1.178 -> 192.168.1.56 TCP 59398 > 21 [ACK] 
0.100249 192.168.1.56 -> 192.168.1.178 TCP 38309 > 113 [SYN] 
0.100285 192.168.1.178 -> 192.168.1.56 TCP 113 > 38309 [RST, ACK] 
0.102590 192.168.1.56 -> 192.168.1.178 FTP Response: 220 ProFTPD 
0.105328 192.168.1.178 -> 192.168.1.56 FTP Request: USER anonymous 


1.139595 192.168.1.178 -> 192.168.1.56 FTP Request: PORT 


1.142317 192.168.1.178 -> 192.168.1.56 FTP Request: LIST 
1.146569 192.168.1.56 -> 192.168.1.178 TCP 20 > 32777 [SYN] 
1.146617 192.168.1.178 -> 192.168.1.56 TCP 32777 > 20 [SYN, ACK] 
1.148892 192.168.1.56 -> 192.168.1.178 TCP 20 > 32777 [ACK] 
1.150032 192.168.1.56 -> 192.168.1.178 FTP Response: 

150 Opening ASCII mode data connection for file list 
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ETP PASV Command 


$ tethereal -n -i wlan0 host 192.168.1.56 
0.100943 192.168.1.56 -> 192.168.1.178 FTP Response: 220 ProFTPD 


1.196822 192.168.1.178 -> 192.168.1.56 FTP Request: PASV 
1.199359 192.168.1.56 -> 192.168.1.178 FTP Response: 

227 Entering Passive Mode (192,168,1,56,149,176). 
1.200114 192.168.1.178 -> 192.168.1.56 TCP 32779 > 38320 [SYN] 
1.202345 192.168.1.56 -> 192.168.1.178 TCP 38320 > 32779 [SYN, ACK] 
1.202381 192.168.1.178 -> 192.168.1.56 TCP 32779 > 38320 [ACK] 
1.203063 192.168.1.178 -> 192.168.1.56 FTP Request: LIST 
1.206525 192.168.1.56 -> 192.168.1.178 FTP Response: 

150 Opening ASCII mode data connection for file list 
1.210180 192.168.1.56 -> 192.168.1.178 FTP-DATA FTP Data: 681 bytes 


@ centeris ae 


Passive FTP and Firewalls 


$ ncftp ftp.plainjoe.org 

Connecting to 192.168.1.56... 

ProFTPD 1.2.10 Server (ProFTPD) [192.168.1.56] 
Logging in... 

Anonymous access granted, restrictions apply. 
Logged in to ftp.plainjoe.org. 

ncftp / > 1s 

Data connection timed out. 


$ tethereal -n -i wlan0 host 192.168.1.56 
0.100943 192.168.1.56 -> 192.168.1.178 FTP Response: 220 ProFTPD 


2.148186 192.168.1.178 -> 192.168.1.56 FTP Request: PASV 
2.150579 192.168.1.56 -> 192.168.1.178 FTP Response: 

227 Entering Passive Mode (192,168,1,56,149,170). 
2.150916 192.168.1.178 -> 192.168.1.56 TCP 32778 > 38314 [SYN] 
5.149642 192.168.1.178 -> 192.168.1.56 TCP 32778 > 38314 [SYN] 

11.148730 192.168.1.178 -> 192.168.1.56 TCP 32778 > 38314 [SYN] 
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DHCP 


€ Protocol for assigning client IP addresses and related 
parameters (DNS servers, netmask, etc...) 
a bootpc (68/udp) 
a bootps (67/udp) 

€ DHCP/BOOTP gateways forward broadcast requests 
(255.255.255.255) to a central server 


€ Simple protocol. What could possibly go wrong? 


@ centeris eda pq 


Example DHCP Session 


# tethereal 


0.000000 


-n -i ethO port 67 or port 68 
Capturing on eth0 
0.0.0.0 -> 255.255.255.255 

DHCP DHCP Discover - Transaction ID 0x7ae74002 
0.090627 192.168.1.56 -> 255.255.255.255 


DHCP DHCP Offer - Transaction ID 0x7ae74002 
0.092209 0.0.0.0 -> 255.255.255.255 
DHCP DHCP Request - Transaction ID 0x7ae74002 


0.217145 192.168.1.56 -> 255.255.255.255 


DHCP DHCP ACK 


@ centeris 


- Transaction ID 0x7ae74002 
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No Lease? 


Listening on 
Sending on 

Sending on 

DHCPDISCOVER 
DHCPDISCOVER 
DHCPDISCOVER 
DHCPDISCOVER 
DHCPDISCOVER 


# dhclient eth0 
LPF/eth0/00:0c:29:3c:69:d2 
LPF/eth0/00:0c:29:3c:69:d2 
Socket/fallback 


on 
on 
on 
on 
on 


etho 
etho 
etho 
etho 
etho 


No DHCPOFFERS received. 


to 255.255.255.255 port 67 interval 7 

to 255.255.255.255 port 67 interval 10 
to 255.255.255.255 port 67 interval 14 
to 255.255.255.255 port 67 interval 19 
to 255.255.255.255 port 67 interval 11 


@ centeris 


# tethereal -n -i wlan0 port 67 or port 68 
0.000000 0.0.0.0 -> 255.255.255.255 DHCP 


7.428451 0.0.0.0 -> 255.255.255.255 DHCP 


19.376876 0.0.0.0 -> 255.255.255.255 DHCP 


DHCP Discover - Transaction ID 0xc065646a 


DHCP Discover - Transaction ID 0xc065646a 


DHCP Discover - Transaction ID 0xc065646a 
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Dead DHCP Server 


# nmap -sU -p 67 -PI -PT 192.168.1.56 


Starting nmap 3.81 ( http://www.insecure.org/nmap/ ) 
Interesting ports on thorn.plainjoe.org (192.168.1.56): 
PORT STATE SERVICE 

67/udp closed dhcpserver 

MAC Address: 00:02:E3:14:D7:91 (Lite-on Communications) 


Nmap finished: 1 IP address (1 host up) scanned in 0.394 seconds 


4 tethereal -n -i wlan0 host 192.168.1.56 
0.532889 192.168.1.178 -» 192.168.1.56 
BOOTP [Malformed Packet] 
0.534637 192.168.1.56 -> 192.168.1.178 
ICMP Destination unreachable (Port unreachable) 
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No Available Leases 


# nmap -sU -p 67 -PI -PT 192.168.1.56 


Starting nmap 3.81 ( http://www.insecure.org/nmap/ ) 
Interesting ports on thorn.plainjoe.org (192.168.1.56): 
PORT STATE SERVICE 

67/udp open|filtered dhcpserver 

MAC Address: 00:02:E3:14:D7:91 (Lite-on Communications) 


Nmap finished: 1 IP address (1 host up) scanned in 0.394 seconds 


# tethereal -n -i wlan0 host 192.168.1.56 

0.114482 192.168.1.178 -> 192.168.1.56 
BOOTP [Malformed Packet] 

0.215411 192.168.1.178 -> 192.168.1.56 
BOOTP [Malformed Packet] 
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HTTP 


€ TCP based (port 80) 

4 SSL enabled servers usually configured on port 443/tcp 

€ Generally simple text based command language 

3 GET, PUT, POST, AUTH, etc.... 

€ Three digit error codes similar to FTP and SMTP 

4 200 OK, 404 page not found, etc... 

€ Heavily used servers can deal with thousands of 
clients 
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HTTP Statistics 


* IPv4 Conversations: web.pcap "x 
IPv4 Conversations 
Address A Address B [Packets Bytes” Packets A->B |Bytes A->B | Packets A<-B | Bytes A<-B " 
-—56.70.73.150 151.47.133.43 4732 3651417 2348 3502297 2384 149120 — chatty 
66.70.73.150 222.248.235.163 636 Ü 46402 1025 69960 . 
66.70.73.150 80.254.182.46 705 634036 428 607351 277 26685 clients 
66.70.73.150 85.220.32.4 728 626283 443 600732 285 25551 
66.70.73.150 68.144.218.17 488 470923 317 458043 171 12880 
66.70.73.150 219.37.72.103 419 367114 251 356108 168 11006 
66.70.73.150 82.35.58.194 599 366852 272 332903 327 33949 n 
AR 7n 73 150 104 2299 11 12& 412 357794 2&n 243550 162 14224 - 
"Copy ) 
X Close 
fropic / Item Yount) Rate Percent E 


> HTTP Request Packets 788 0.002318 6.91% 
= HTTP Response Packets 676 0.001988 5.9396 


???: broken 0 0.000000 0.00% 

1xx: Informational 0 0.000000 0.0096 
> 2xx: Success 612 0.001800 90.53% 
> 3xx: Redirection 9-900129 6.51% 
7 4xx: Client Error 20 0.000059 2.96% 


404 Errors : 


404 Not Found 20 


Evy: Carvar Errar n nnnnnn n nno 


X Close 


@ centeris Re IN E 


HTTP 404 


web.pcap - Ethereal <2> SUN 


File Edit View Go Capture Analyze Statistics Help 


Filter all HTTP 
responses for 
“page not found” 


Expression... Clear) V Apply | 


Protocol Info 


Hypertext Transfer Protocol 
Line-based text data: text/html 


> Frame 7745 (491 bytes on wire, 491 bytes captured) 

> Ethernet II, Src: Supermic 12:27:05 (00:30:48:12:27:05), Dst: All-HSRP-routers 00 (00:00:0c:07:ac:00) 

> Internet Protocol, Src: 66.70.73.150 (66.70.73.150), Dst: 206.70.251.252 (206.70.251.252) 

> Transmission Control Protocol, Src Port: http (80), Dst Port: 28062 (28062), Seq: 1013802675, Ack: 30778842¢ 
b 

b 


Reassemble 
fragmented HTML 


45 00 Jaj 
ce 46 

80 18 

5687 ! 

20 4e "&HTTP/1 .1 404 

3a 20 ot Found ..Date: 

30 35 Mon, 31 Oct 2005 

0a 53 12:22:2 3 GMT..S 


response 


File: "web.pcap" 16 MB 0070 


@ centeris 
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TCP Streams 


Analyze -> Follow TCP Stream 


* 


iStream Content 


| GET /documentation.html HTTP/1.1 
| Host: rsync.samba.org 

| Connection: keep-alive 

Accept: */* 


Accept-Encoding: gzip, deflate 

User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; 
X-Forwarded-For: 206.70.248.253 

Via: 1.1 pnetcache (NetCache NetApp/5.3.1R3) 


HTTP/1.1 200 OK 

Date: Mon, 31 Oct 2005 12:22:05 GMT 
Server: Apache 

| Keep-Alive: timeout=15, max=100 

| Connection: Keep-Alive 

| Transfer-Encoding: chunked 

| Content-Type: text/html 


ec9 


DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN"> 
«HTML» 
| «HEAD» 
| <TITLE>rsyne documentation</TITLE> 
| </HEAD> 


style="margin-top: 0"> 


Follow TCP stream 


Accept-Language: en-us,rsl 7a9c4aBeflf;q-0.0,rs2 d4f8c943825;q-0.0,rs3 2a2f8d4db8;q-0.0 


NET CLR 1.1.4322) 


«BODY BGCOLOR-"&Rffffff" TEXT="#000000" VLINK="#292555" LINK="#292555" ALINK-"$cc0033" 


—————— O) = 


| (Gisave As)(& Print) Entire conversation (8102 bytes) 


ve ASCII © EBCDIC © Hex Dump © C Arrays © Raw 


) 


X Close 


Filter out this stream) C 


@ centeris 
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Dealing with Encryption 


€ ssldump — protocol analyzer for SSL/TLS traffic 
2 http://www.rtfm.com/ssldump/ 


$ ssldump -n -q -r /tmp/dump.pcap host 192.168.1.178 


New TCP connection #1: 192.168.1.178(48688) <-> 66.70.73.153 (443) 


11 0.0356 (0.0356) C»S SSLv2 compatible client hello 

12 0.1018 (0.0662) S>C Handshake ServerHello 

13 0.1018 (0.0000) S>C Handshake Certificate 

14 0.1018 (0.0000) S>C Handshake ServerKeyExchange 
15 0.1018 (0.0000) S>C Handshake ServerHelloDone 


New TCP connection #2: 192.168.1.178(48689) <-> 66.70.73.153 (443) 
21 0.1837 (0.1837) C>S Handshake ClientHello 

1 28 1.9144 (0.2089) S>C application_data 

1 29 1.9168 (0.0024) C>8$ application data 


2 2 0.2391 (0.0553) S>C Handshake ServerHello 
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Decoding SSL/TLS 


€ ssldump can decode application data if provided with 
the appropriate key 


$ ssldump -k ssl.key -dqn -r /tmp/dump.pcap host 192.168.1.178 
New TCP connection H1: 192.168.1.178(46915) <-> 66.70.73.153 (443) 


1 10 0.2637 (0.1237) C>S application data 
GET / HTTP/1.1 
Connection: Keep-Alive 
User-Agent: Mozilla/5.0 (compatible; Konqueror/3.4; Linux) 
KHTML/3.4.0 (like Gecko) 
Accept: text/html, image/jpeg, image/png, text/*, image/*, */* 
Accept-Encoding: x-gzip, x-deflate, gzip, deflate 
Accept-Charset: utf-8, utf-8;q=0.5, *;q=0.5 
Accept-Language: en 
Host: bugzilla.samba.org 


@ centeris pe 


NFS 


€ Several versions to deal with 

4 NES v1 and v2 used UDP 

4 NFSv3 adds TCP support 

4 NFSv4 latest incarnation 

€ Based on ONC-RPC 

2 services register their port with the RPC portmapper 


9€ eXternal Data Representation (XDR) 
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Why does “mount -t nfs” hang? 


# mount -t nfs worm.plainjoe.org:/export/u2/suse /mnt 


# tethereal -n -i lo 
Capturing on lo 
0.000000 127.0.0.1 -> 127.0.0.1 
Portmap V2 UNSET Call 
0.000694 127.0.0.1 -> 127.0.0.1 
ICMP Destination unreachable (Port unreachable) 
4.998372 127.0.0.1 -> 127.0.0.1 
Portmap [RPC retransmission of #1]V2 UNSET Call 
4.998405 127.0.0.1 -> 127.0.0.1 
ICMP Destination unreachable (Port unreachable) 
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Mismatched NFS Versions 


$ grep NFS /var/log/messages 

Oct 31 09:23:41 fc4 automount [2241]: 
>> mount to NFS server 'ahab.plainjoe.org' failed: 
possible invalid protocol. 


$ tethereal -r dump.pcap -n \ 
-R "portmap && ip.addr -- 192.168.1.113" 


13 0.045103 192.168.1.113 -> 192.168.1.79 
Portmap V2 GETPORT Call NFS(100003) V:3 TCP 

15 0.045585 192.168.1.79 -> 192.168.1.113 
Portmap V2 GETPORT Reply (Call In 13) 
PROGRAM NOT AVAILABLE 
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CIFS/SMB 


€ Utilized by Windows as well as non-Microsoft 
platforms 

€ NetBIOS services vs. TCP+UDP+DNS 

O Active Directory domains utilize many protocols 

4 CIFS, DCE-RPC, LDAP, DNS, Kerberos 5, etc... 

9 More changes coming in Windows Vista 

4 symbolic links, transactions, etc... 

€ Many features are negotiated 

4 e.g. password encryption, 32-bit status codes, extended 
security (SPNEGO), and DFS support 

€ Resources listed at http://devel.samba.org 
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Samba 


€ Windows 2000 client unable to logon to a Samba 
domain 


€ Answer: nmbd is running, but smbd has died 
a connection to port 139 (netbios-ssn) receives a RST 


0.000000 172.16.29.128 -> 172.16.29.255 
NBNS Name query NB RAIN<20> 
0.003109 172.16.29.1 -> 172.16.29.128 
NBNS Name query response NB 172.16.29.1 


0.003883 172.16.29.128 -> 172.16.29.1 

TCP danf-ak2 » netbios-ssn [SYN] 
0.003911 172.16.29.1 -> 172.16.29.128 

TCP netbios-ssn > danf-ak2 [RST, ACK] 
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Network Path Not Found 


c:\> net view \\lettuce 
System error 53 has occurred. 
The network path was not found. 


$ tethereal -r xp_firewall.pcap -R 'nbns' 

17 10.846761 172.16.29.128 -> 172.16.29.255 
NBNS Name query NB LETTUCE<20> 

23 11.594905 172.16.29.128 -> 172.16.29.255 
NBNS Name query NB LETTUCE<20> 

24 12.346497 172.16.29.128 -> 172.16.29.255 
NBNS Name query NB LETTUCE<20> 


@ centeris rn 


Normal Krb5 SMB Session 


Transmission Control Protocol, Src Port: 1091, Dst Port: 445 
NetBIOS Session Service 
SMB (Server Message Block Protocol) 

SMB Header 

Session Setup AndX Request (0x73) 


Security Blob: 608204FB06062B0601050502A08204EF308204EBA030302E... 
GSS-API Generic Security Service Application Program Interface 
OID: 1.3.6.1.5.5.2 (SPNEGO - Simple Protected Negotiation) 

SPNEGO 

negTokenInit 

mechTypes: 4 items 
Item: 1.2.840.48018.1.2.2 (MS KRB5 - Microsoft Kerberos 5) 
Item: 1.2.840.113554.1.2.2 (KRB5 - Kerberos 5) 

Item: 1.2.840.113554.1.2.2.3 (KRB5 - Kerberos 5 - User to User) 
Item: 1.3.6.1.4.1.311.2.2.10 (NTLMSSP) 

mechToken: 608204AD06092A864886F71201020201006E82049C308204... 
krb5 blob: 608204AD06092A864886F71201020201006E82049C308204... 

Native OS: Windows 2000 2195 

Native LAN Manager: Windows 2000 5.0 

Primary Domain: 
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AD and NTLMSSP Fallbacks 


€ Windows client do not obtain a service ticket when 
accessing a server using an IP in the UNC path 


Transmission Control Protocol, Src Port: 1106, Dst Port: 445 
NetBIOS Session Service 
SMB (Server Message Block Protocol) 

SMB Header 

Session Setup AndX Request (0x73) 


Security Blob: 604806062B0601050502A03E303CA00E300C060A2B060104... 
GSS-API Generic Security Service Application Program Interface 
OID: 1.3.6.1.5.5.2 (SPNEGO - Simple Protected Negotiation) 
SPNEGO 
negTokenInit 
mechTypes: 1 item 
Item: 1.3.6.1.4.1.311.2.2.10 (NTLMSSP) 
mechToken: 4E544C4D5353500001000000978208E20000000000000000... 
NTLMSSP 


@ centeris evo ps 


LDAP 


€ Nine core operations 

4 bind, unbind, abandon 

3 search, compare 

4 modify, add, delete, moddn 

4 plus extended operations and controls 

9€ Generally string based with some LBER encoding 

9€ Does include data privacy via 

4 LDAPS protocol 

4 StartTLS extended operation 

O Distributed directories are linked together via 
referrals and references 
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Where can things go wrong? 


9 Invalid authentcation 
Q clear text binds, Kerberos ticket failures, etc... 
€ Referrals to unknown servers 
€ Misconfigured search filters 
9 Failures in extended operations or controls when 
marked as critical 
O Limits on search result sizes or durations 
O Missing attributes or object classes 
€ Misconfigured access control lists 
€ Corrupted indexes 
9 blah, blah, blah, ... 


@ centeris ae 


LDAP Search 


€ Each result is returned in a single PDU 


$ ldapsearch -h ldap.plainjoe.org -x \ 
-b 'dc-plainjoe,dc-org' '(uid=jerry)' objectclass 


dn: uid-jerry,ou-people,dc-plainjoe,dc-org 
objectClass: inetOrgPerson 

objectClass: posixAccount 

objectClass: sambaSamAccount 

objectClass: sambaIdmapEntry 


8 0.010113 192.168.1.178 -> 192.168.1.56 


9 0.014480 192.168.1.56 -> 192.168.1.178 
LDAP MsgId=2 Search Entry 

10 0.014981 192.168.1.56 -> 192.168.1.178 
LDAP MsgId=2 Search Result 


# tethereal -n -r ldap_search.pcap -R "ldap.message_id == 2" 


LDAP MsgId-2 Search Request, Base DN-dc-plainjoe,dc-org 
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LDAP Search Result 


Transmission Control Protocol, Src Port: 389 (389), 

Dst Port: 56763 (56763), Seq: 1088281708, 

Ack: 2698029667, Len: 129 

Lightweight Directory Access Protocol 

LDAP Message, Search Entry 
Message Id: 2 
Message Type: Search Entry (0x04) 
Message Length: 122 
Response To: 8 
Time: 0.003292000 seconds 


Attribute: objectClass 
Value: inetOrgPerson 
Value: posixAccount 
Value: sambaSamAccount 
Value: sambaIdmapEntry 


Distinguished Name: uid-jerry,ou-people,dc-plainjoe,dc-org 
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